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ABSTRACT 


The  purpose  of  this  study  was  to  develop  physical  and  selection  performance  standards  for  the  screening  of 
candidates  for  entrance  into  the  Unmanned  Aerial  Vehicle  (UA\0  Pioneer  Pilot  training  program.  A  minimum 
Pioneer  crew  consists  of  an  external  pilot,  internal  pilot,  and  a  mission  commander/payload  specialist.  The 
mission  commander/payload  specialist  is  responsible  for  the  overall  planning  and  execution  of  the  specific  mission 
and  control  of  the  visu^information  gathering  during  the  mission.  The  internal  pilot  is  responsible  for  the  control 
of  the  Pioneer  when  it  is  beyond  visual  range.  The  external  pilot  is  responsible  for  take-offs,  landings,  and  any  in¬ 
visual-range  control  of  the  vehicle.  A  task  analysis  was  done  in  the  training  and  fleet  squadrons  to  identify  critical 
tasks  for  safe  flight  and  the  relevant  skills  required  to  perform  the  piloting  tasks.  From  this  task  analysis,  specific 
computer-based  tests  batteries  were  chosen  as  potential  predictor  variables.  The  system  was  programmed  and 
students  and  external  pOots  were  administered  the  test  battery.  A  composite  training  measure  was  created  fi-om 
objective  training  scores,  verified  with  subjective  instructor  ratings,  and  used  as  the  criterion  for  predictive 
validation  of  the  system.  The  sample  size  was  small  for  the  preliminary  model,  but  a  significaiit  relationship 
between  a  composite  of  multitask  tracking  scores  and  UAV  performance  was  observed  (adjusted  =  0.86).  In 
^HHitinn  Structured  and  unstructured  interviews  of  the  Pioneer  crews,  students,  instructors  and  senior  squadron 
per^nnel  were  used  to  identify  important  physical  characteristics  essential  for  safe  operation  of  the  Pioneer.  These 
traits  were  then  used  to  derive  medical  screening  criteria  for  all  crew  positions. 
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INTRODUCTION 


Unmaimed  Aerial  Vehicles  (UAVs)  are  remotely  piloted  vehicles  (RPVs)  currently  used  by  the  U.S.  Navy, 
Marine  Corps,  Army,  and  Air  Force  for  airborne  reconnaissance,  surveillance,  target  acquisition,  fire  support 
adjustment,  and  battle  damage  assessment.  The  U.S.  Army  and  Air  Force  are  currently  involved  with  the  test  and 
evaluation  of  a  number  of  larger  UAV  ^sterns  such  as  the  Predator  and  Hunter  systems.  These  ^sterns  can  remain 
airborne  for  long  periods  of  time,  have  high  altitude  capabilities,  and  carry  relatively  heavy  payloads.  They  also 
reqitire  lengthy  runw^  for  take  off  and  landing  and  larger  support/maintenance  crews  for  operation.  The  Navy, 
due  to  its  unique  operational  environment,  is  currently  using  the  smaller  Pioneer  UAV  for  RPV  support  in  the 
field.  The  Pioneer  is  a  relatively  small  air  vehicle  with  a  wing  q)an  of  16.9  ft,  a  length  of  14  ft,  and  a  height  of  3.3 
ft  and  is  capable  of  providing  information  to  amphibious  forces  without  over-taxing  a  ship’s  storage  capacity 
and/or  necessitating  the  removal  of  other  airborne  and  warfighting  assets  aboard  ship.  For  shipboard  operations, 
the  Pioneer  can  be  laimched  directly  firom  a  stand  (called  a  rocket  assisted  take  off  or  RATO)  fixjm  the  deck  of  the 
ship  and  can  be  recovered  by  a  catch  net  on  the  bow  of  the  ship.  These  nets  are  suitable  for  both  daytime  and 
nighttime  operations.  This  method  of  launch  and  recovery  obviates  the  need  for  carrier  decks  for  this  vehicle.  The 
portable/mobile  Pioneer  systems  can  also  be  used  in  the  field  to  support  deployed  Marine  Corps  units,  or  launched 
fi’om  a  general  purpose  amphibious  assault  ship  and  ‘passed’  off  to  portable  field  imits  for  reconnaissance.  Field 
take  offe  and  landing  require  as  little  as  21  meters  for  pneumatic  launches  and  70  meters  for  recovery. 

The  Pioneer  contains  an  expensive  electronics  package  and  surveillance  equipment,  with  the  total  airborne 
system  costing  in  excess  of  $800,000.  The  Pioneer  can  relay  video  and/or  telemetry  information  in  real  time  to  a 
groimd  control  station  and/or  portable  control  units  for  deployed  forces.  Its  service  ceiling  is  12,000  ft,  with  a 
cruising  speed  of  65  kts  and  a  maximum  range  of  185  km.  With  its  low  radar  cross-section,  low  infiared  signature, 
and  remote  control  versatility,  the  system  is  ideal  for  urunarmed  battle  damage  assessment  and  search  and  rescue 
operations  in  hostile  areas.  The  Pioneer  ^stem  and  crews  were  successfully  deployed  and  used  in  support  of 
operations  during  the  Gulf  War. 

The  minimum  persormel  requirements  for  operation  of  the  Pioneer  is  a  three  person  crew:  an  external  pilot 
(EP),  an  internal  pilot  (IP),  and  a  mission  cormnander/payload  operator.  Some  operations  may  also  include  an 
intelligence  officer  as  a  crew  member.  The  primary  responsibility  of  the  IP  is  flight  control  of  the  aircraft  using 
only  instrumentation  when  the  craft  is  b^ond  visual  range  and  performing  its  mission.  Most  of  this  flying  is  done 
on  autopilot.  When  in  visual  range,  and  during  take-offs  and  landings,  the  aircraft  control  is  handed  off  to  the  EP 
while  the  IP  feeds  him/her  aircraft  speed,  heading  and  altitude  information.  Both  the  IP  and  EP  positions  are  held 
by  petty  officers  or  nonconunissioned  officers  with  prior  aviation-related  experience  (i.e.,  maintenance  technicians, 
etc.).  The  mission  commander  is  responsible  for  the  overall  safety  and  success  of  the  mission,  and  the  payload 
operator  controls  the  visual  sensors  during  flight.  Mission  conunanders  are  most  often  officers  with  prior  aviation 
experience.  All  mission  commanders  must  be  IP  qualified.  In  addition  to  their  primary  duties,  all  crew  members 
are  responsible  for  system  assembly  and  breakdown,  and  minor  maintenance  and  repair  of  the  aircraft  when  it  is 
deployed.  All  crew  members  receive  their  initial  training  at  the  Department  of  Defense  UAV  Training  Center 
(DUTC).  This  initial  training  includes,  at  a  minimum,  a  groimd  syUabus  on  tystems,  basic  operations,  and 
procedures  of  the  Pioneer,  as  well  as  actual  flight  training  with  sinqrle  RPVs,  scaled-down  versions  of  the  Pioneer, 
and  then  the  actual  Pioneer  system.  Despite  the  complex  p^chomotor,  cognitive,  and  perceptual  skills  required  to 
control  this  aircraft,  no  a  priori  selection  (medical  or  performance-based)  criteria  were  established  for  entry  into 
this  training  curriculum. 

The  EP  initial  program  includes  an  extensive  18-week  tyllabus,  with  a  week  of  ground  school  and  then  flight 
training  for  the  remainder  of  the  tyllabus.  Following  completion  of  DUTC  training,  graduates  are  sent  to  either  a 
Marine  Corps  unit  or  a  Navy  squadron  detachment  for  further  hands-on  training  before  deployment  with 
operational  forces.  In  the  past,  students  who  were  marginal  performers  at  DUTC  were  given  extended  training  (up 
to  22  weeks)  to  meet  minimum  accepted  standards  before  to  being  sent  to  their  units  On  occasion,  the  field  units 
have  found  that  these  students  arrive  without  all  of  the  requisite  skills/abilities  to  be  safely  deployed  to  the  high- 
stress,  highly  demanding  operational  environment.  In  adffition,  recent  mishaps  that  included  the  complete  loss  of 
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the  aircraft  due  to  hiunan  error  led  to  the  decision  by  the  Navy’s  UAV  community  to  try  and  improve  the  caliber  of 
applicants  who  enter  training  and,  thereby,  reduce  training  costs  and  maintain  the  quality  of  the  pilots  in  the  fleet. 

The  Naval  Aerospace  Medical  Research  Laboratory  (NAMRL)  was  tasked  by  Naval  Air  Systems  Command 
(NAVAIR)  to  develope  a  performance-based  selection  system  for  predicting  which  UAV  EP  candidates  have  the 
greatest  probability  of  (1)  completing  UAV  EP  training,  (2)  completing  UAV  EP  training  within  a  shorter  time 
period,  and/or  (3)  demonstrating  higher  proficiency  in  performance  of  EP  tasks  following  training.  An  additional 
tasking  from  NAVAIR  was  to  enlist  the  aid  of  the  Aeromedical  Advisory  Council  at  the  Naval  Operational 
Medicine  Institute  (NOMI)  to  develop  medical  standards/qualifications  for  UAV  EPs  and  IPs.  The  need  for 
screening  medical  standards  was  evident  in  a  mrmber  of  cases  where  individuals  without  the  physical  means  (i.e., 
grip  strength  problems,  night  blindness,  etc.)  to  complete  training  were  nonetheless  sent  to  DUTC  for  the  18-week 
syllabus.  Due  to  the  small  size  of  this  community,  any  delay  attrition  from  training  severely  limits  the  capability  of 
the  fleet  squadrons  to  meet  their  operational  requirements. 

METHODS 

The  methods  used  to  develop  and  validate  the  UAV  selection  system  and  medical  standards  included  field 
observations,  expert  and  student  interviews,  curriculum  reviews,  and  standard  testing  and  psychometric  evaluation 
of  the  test  battery  scores  and  criterion  data. 

SUBJECTS 

During  the  task-analysis  phase  of  the  study,  currently  qualified  and  student  EPs,  IPs,  and  Mission 
Commander/payload  operators  were  observed  and  interviewed  for  critical  phases  of  flight  and  the  essential 
skills/physical  abilities  needed  to  perform  their  jobs.  Instructors  and  senior  squadron  personnel  (OICs,  Safety 
Officers,  etc.)  were  also  interviewed.  These  observations  were  made  at  the  initial  training  site  at  DUTC  in  Ft. 
Huachuca,  Arizona,  at  VC-6  detachment  at  Patuxtant  River,  Maryland,  and  at  VMU  1  at  29  Palms,  California.  For 
the  system  validation  portion  of  the  stucfy,  eight  student  EPs  were  tested  at  DUTC,  and  six  current  EP/IPs  were 
tested  at  VC-6  detachment.  All  participants  were  right-handed  males;  the  overall  average  age  was  29.57  {SD  = 
5.24).  The  average  age  for  the  students  was  27.5  {SD  =  4.72),  and  the  average  age  of  the  pilots  was  31.17  {SD  = 
5.91). 

APPARATUS 

The  Computer-based  Performance  Test  (CBPT)  is  a  2-h  test  battery'  that  measures  eye-hand-foot  coordination 
(multitasking),  spatial/mental  ability,  divided  auditory  attention,  and  cognitive  skills.  Similar  to  a  video  game  run 
on  an  IBM-  or  PC-compatible  computer  (386/25  MHZ  or  greater),  the  system  requires  a  VGA  monitor,  two 
joysticks,  rudder  pedals,  numeric  keypad,  and  headphones.  The  CBPT  has  several  prograrruned  test  batteries  that 
are  predictors  of  student  naval  aviator  and  naval  flight  officer  performance  and  attrition  (Delaney,  1992;  Street, 
Chapman,  &  Helton,  1993).  The  system  has  also  been  used  for  the  Landing  Craft  Air  Cushion  Vehicle  operators 
and  engineer  position  training  outcome  (Dolgin  &  Nontasak,  1990;  Nontasak,  Dolgin,  Blower,  1991;  Ro^rtson  & 
Nontasak,  1996).  In  addition,  one  new  test  was  created  and  prograrruned  for  this  project. 

PROCEDURE 

Critical  tadcs  for  the  safe  operation  of  UAVs  in  the  operational  environment  were  identified  through  structured 
and  unstructured  interviews  with  instructors,  EPs,  and  IPs,  as  well  as  observations  of  actual  flight  training  and 
field  operatiorrs  using  the  Pioneer.  The  identified  critical  tasks  were  analyzed  for  the  required  behavioral  skills  and 
minimum  physical  standards.  Subtests  from  the  CBPT  that  measure  these  identified  skills  were  chosen  as  potential 
predictors  and  included  in  the  test  battery.  Criterion  measures  were  identified  via  curriculum  reviews  and 
instructor  interviews. 
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Critical  Task  and  Task  Analysis:  Performance 

Analysis  of  Pioneer  field  operations  highlighted  the  importance  of  a  few  critical  time  periods  during  flight. 
Barring  a  systems  failure,  instrument  control  by  the  IP  is  adequate  to  maintain  safe  flight  when  the  craft  is  outside 
of  visual  range.  In  visual  range,  critical  junctures  occur  during  the  transfer  of  control  (IP  to  EP),  following  wave- 
offs  when  control  reversals  occur,  and  during  landings.  For  example,  during  shipboard  landings,  the  Pioneer  is 
targeted  to  the  center  of  a  vertical  net  located  on  the  ship.  The  target  zone  is  a  section  of  net  approximately  60  x  20 
ft.  For  night  missions,  these  c<q)tures  are  accomplished  by  darkening  the  lights  on  the  flight  deck  and  illumirrating 
the  central  portion  of  the  net.  The  EP  then  estimates  aircraft  distance  and  direction  fiom  the  net  by  the  relative 
location  and  distance  between  the  port  and  starboard  lights  on  the  aircraft  wings.  Typically,  EPs  place  an  earcup  on 
one  ear  and  leave  the  other  ear  uncovered  during  this  and  all  other  in-visual-range  tasks.  This  enables  them  to 
attend  simultaneously  to  the  heading  and  airspeed  calls  of  the  IP,  as  well  as  to  the  shipboard  sounds/environment. 
Although  the  Marine  Corps’  operational  envirorunent  differs  from  that  of  the  Navy,  a  similar  method  of  one  ear 
covered,  one  imcovered  has  been  adopted  to  maintain  situational  awareness.  In  addition,  the  Marines  experience 
the  iciniR  critical  junctures  during  IP  to  EP  transfer,  control  reversals  diuing  aborted  landings/fly-bys,  and 
nighttime  arrested  landings.  The  tasks/skills  necessary  for  EPs  to  successfully  complete  these  maneuvers  include 
mental  reversals/rotation,  time  estimation  to  contact  for  the  approaching  aircraft,  hand-eye  coordination,  selective 
auditory  attention^  and  multitasking  (psychomotor  and  visual  components).  For  the  IP,  mission  commander,  and 
payload  operator,  no  critical  skills  or  problem  areas  were  apparent  nor  seemed  to  be  relevant  for  screening 
pirrposes.  In  addition,  IP  training  is  only  a  2-month  curriculum,  and  preventable  mishaps  and  difBculties  with 
personnel  at  the  fleet  level  have  focused  on  the  EP  position. 

The  following  tasks  were  chosen  for  the  experimental  test  battery  due  to  their  correspondence  with  the  above- 
mentioned  skills  The  battery  consists  of  three  major  categories: 

1.  Psychomotor  (PMT)  and  Dichotic  Listening  (DLT)  Tests 

2.  Horizontal  Tracking  (HT)  and  Digit  Cancellation  (DC)  Tests 

3.  Manikin  and  Time  Estimation  Task  (TET)  Tests 

1.  PMT  and  DLT  Tests.  The  subtasks  within  this  test,  and  the  order  in  which  they  were  adnrinistered, 
are  listed  below: 

A.  Stick  Task:  The  sulqects  use  the  joystick  (referred  to  as  the  stick  )  directly  in  front  of  them  to  control 
the  green,  middle  cursor  (shaped  like  a  cross  “+”)  on  the  screen.  The  object  of  the  tadc  is  to  maintain  the  cursor  at 
the  center  of  the  screen  where  the  vertical  and  horizontal  rows  of  dots  intersect  (see  Figure  1).  Movement  of  the 
stick  causes  the  cursor  to  move  in  the  opposite  direction,  and  the  natural  drift  of  the  cursor  (if  the  stick  is  released) 
is  toward  the  upper  right-hand  comer  of  the  screen.  The  subjects  have  one  3-min  practice  session,  followed  by  a 
brief  pause  before  the  actual  scored  test  starts. 
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Figure  1.  PMT:  Stick  Task. 

(cursor  at  zero  error  position) 


B.  DLT  Task:  In  this  test,  the  subjects  are  presented  a  unique  string  of  numbers  and  letters  to  each  ear 
simultaneously.  The  computer  specifies  which  ear  to  pay  attention  to  (“RIGHT’  or  “LEFT”).  Using  their  left  hand, 
the  subjects  key  in  only  the  numbers  heard  in  the  designated  ear  while  ignoring  the  letters.  The  DLT  consists  of  12 
trials.  In  each  trial,  the  subjects  receive  two  different  streams  of  information  with  a  pause  between  them.  The  first 
stream  contains  five  numbers  mixed  with  letters.  The  second  stream  contains  four  numbers,  also  mixed  with 
letters.  An  example  of  a  DLT  trial  is  illustrated  in  Table  1.  In  this  table,  the  Left  and  Right  Ear  Stimulus  columns 
represent  the  spoken  messages  that  the  subject  hears  in  each  ear.  The  rows  of  the  table  represent  the  sequence  in 
time. 


Table  1.  DLT  Trial  Visual  Example. 


:;:Sequeacc 

Ldt  EariStiimiius 

:Ri^  Ear  Stimulus 

Trial# 

“Test  1" 

“festr 

Attention 

“RIGHT 

“RIGHT 

Stimuli 

Pause 

“R8NSMY2GB7FL6RL5” 

“YL3SR4FZ9XF0FN1L" 

Attention 

“LEFT 

“LEFT 

Stimuli 

“B  F  4  3  7  9” 

“G  L  1  5  6  2" 

Because  subjects  often  find  the  DLT  difficult  to  initially  comprehend,  they  are  given  four  practice  sessions 
before  beginning  the  actual  test.  Both  the  correct  responses  and  the  subjects'  responses  appear  on  the  screen  after 
each  practice  session. 

C.  Stick  &  DLT  Task:  In  this  test,  subjects  combine  the  STICK  and  DLT  tasks.  The  stick  tracking  task  is 
performed  with  the  right  hand  grasping  the  stick,  and  the  DLT  responses  are  keyed  in  with  the  left  hand  on  the 
numeric  keypad.  This  task  has  no  DLT  practice  sessions.  The  DLT  begins  after  about  15  seconds  into  the  tracking 
task,  and  12  trials  are  presented.  After  the  last  DLT  trial,  the  subject  continues  the  tracking  task  until  the  session 
terminates. 


4 


D.  Stick  &  Rudder  Task:  The  subject  performs  the  rudder  (the  pedals)  task  and  the  stick  task 
simultaneously.  The  rudder  cursor  moves  side  to  side.  The  task  is  to  use  the  rudder  pedals  to  hold  the  cursor  on  the 
vertical  line  (on  the  lower  left  center  of  the  screen)  while  maintaining  the  other  cursor  on  the  appropriate  target. 
The  cursor  moves  back  and  forth  in  the  same  direction  as  the  rudder  movement. 

E.  Stick,  Rudder,  and  DLT  Task:  The  participant  performs  the  stick,  rudder,  and  DLT  tasks 
simultaneously.  Subjects  perform  DLT  inputs  are  made  with  the  left  hand. 

F.  Stick,  Rudder,  and  Throttle  Task:  The  participant  performs  the  throttle  (the  joystick  to  the  left),  stick, 
and  rudder  tasks  simultaneously.  The  red  throttle  cursor  moves  up  and  down.  The  task  is  to  use  the  left  hand  to 
hold  the  throttle  joystick  to  keep  the  cursor  on  the  horizontal  Une  (on  the  left  side  of  the  screen)  while  maintaining 
the  other  cursors  on  the  appropriate  targets.  The  left  cmrsor  moves  rqr  and  down  in  the  same  direction  as  the 
throttle  movement.  Figure  2  depicts  the  cursors  at  the  zero-error  position  for  this  task.  These  combined  tasks  have 
all  been  found  to  be  good  models  of  mirlti-tasking/high  task  loads  and  correspond  to  the  multitasking  situations 
encountered  routinely  by  EPs. 


Figure  2.  PMT:  Stick,  Rudder,  and  Throttle  Task. 


Table  2  illustrates  the  sequence  and  duration  of  the  tests  that  make  up  the  PMT/DLT  portion  of  the  test 
battery.  Generally,  1  hour  is  needed  to  complete  all  PMT/DLT  activities.  The  PMT/DLT  tests  were  chosen  for  the 
experimental  battery  due  to  their  correspondence  with  the  hand/eye  coordination  and  multitasking/workioad 
requirements  of  the  EP,  as  well  as  their  mode  of  communication  in  operational  environments  (one  ear  covered,  one 
uncovered). 
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Table  2.  PMT/DLT  Test  Duration. 


HiTiest  M. . 

:  .-Trial 

[ime  (miii) 

1.  Stick 

Practice 

3 

3 

Scored 

3 

6 

2.DLT 

Practice 

3 

9 

Scored 

4 

13 

3.  Stick/DLT  Dual 

Scored 

4 

17 

4.  Stick  &  Rudder 

Practice 

3 

20 

1st  Scored 

3 

23 

2nd  Scored 

3 

26 

5.  Triple  (stick,  rudder,  DLT ) 

Scored 

4 

30 

6.  Throttle  (with  stick,  and  rudder) 

Practice 

3 

33 

1st  Scored 

3 

36 

2nd  Scored 

3 

39 

2.  The  HT  and  DC  Tests. 

A.  HT  Task:  This  portion  of  the  test  requires  subjects  to  keep  a  square  cursor  centered  in  a  rectangle  (see 
Fig.  3).  The  cursor  is  driven  by  a  computer-programmed  forcing  function  that  accelerates  as  the  distance  from  the 
centerpoint  increases.  Controlling  the  stick  with  their  right  hand,  subjects  make  appropriate  counterbalance 
movements  of  the  cursor.  The  cursor  moves  in  the  same  direction  as  the  movement  of  the  stick.  To  perform  well 
on  this  task  requires  minute  inputs  and  constant  adjustments  to  ‘balance’  the  cursor  on  the  centerpoint.  The 
subjects  woric  through  seven  of  these  sessions. 

B.  DC  Task:  This  task  requires  subjects  to  use  their  left  hand  to  key  in  numbers  1-4  on  a  keypad  as  they 
appear  on  the  screen.  The  DC  task  is  used  primarily  as  a  distractor. 

C  HT/DC  Task:  This  final  portion  of  the  test  requires  subjects  to  perform  the  horizontal  tracking  and 
digit  cancellation  tasks  at  the  same  time.  The  subjects  must  pay  equal  attention  to  each  task.  The  numbers  for  the 
DC  task  are  centered  at  the  top  of  the  HT  rectangle.  The  subjects  work  through  three  sessions.  Again,  the  DC  is 
performed  with  the  left  hand  and  the  HT  with  the  right  hand. 


Figure  3.  Horizontal  Tracking. 

(cursor  at  zero  position) 


The  length  of  the  HT,  DC,  and  Dual  HT/DC  tasks  is  included  in  Table  3.  The  HT/DC  tests  measure  more  fine 
motor  control  ability  with  a  distractor  task  and  correspond  to  the  multi-tasking  and  fine  motor  controls  required  in 
EP  flight  operations. 
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Table  3.  HT/DC  Test  Duration 


Test  i  Time  (min)  :  Camulative  (min) 

1.  Horizontal  Tracking 

15 

15 

2.  Digit  Cancellation 

2 

17 

3.  Dual  HT/DC 

8 

25 

3.  The  Manikin  and  Time  Estimation  Task  Tests. 

A.  The  Manikin  Test:  The  Manikin  test  is  a  test  of  visuospatial  abilities,  consisting  of  48  drawings  of  a 
sailor  holding  a  square  in  one  hand  and  a  circle  in  the  other.  The  sailor  is  dq)icted  either  right-side  up  or  upside 
down,  faring  toward  or  away  from  the  subject.  The  objective  is  to  determine  as  quickly  as  possible  which  hand  is 
holding  the  square.  The  Manikin  test  is  not  timed.  This  test,  in  its  most  basic  form,  measures  the  ability  to 
perform  mental  rotations/reversals  and  was  chosen  for  being  relevant  to  the  EP  control  reversals  during  fly-bys. 

B.  The  Time  Estimation  Task  (TE7)  Test.  The  TET  is  a  test  of  visual  tracking  and  time  estimation 
abilities.  The  subject’s  task  is  to  track  a  moving  target  (an  aircraft)  as  it  moves  at  a  constant  rate  across  the  screen. 
It  is  visible  brifely  after  appearing  on  the  screen  and  then  disappears.  The  goal  of  the  subject  is  to  accurately 
predict  ( by  pressing  the  joystick  trigger)  when  the  aircraft  should  appear  above  the  target.  When  the  subject 
presses  the  trigger,  the  aircraft  reappears  on  the  screen,  and  the  mean  pixel  error  is  recorded.  The  speed  of  the 
aircraft  varies  from  trial  to  trial,  but  the  initial  visible  view  of  the  aircraft  remains  constant  throughout  the  test.  In 
addition  to  the  TET,  the  aforementioned  HT  task  is  included  as  a  distractor  with  this  test.  Thus,  the  subject  must 
maintain  the  cursor  on  the  center  point  with  the  joystick  while  firing  the  joystick  trigger  at  the  appropriate  time. 
The  TET  was  specifically  created  for  this  test  battery.  The  task  measures  time/distance  estimation  abilities  which 
are  essential  to  the  EPs’  control  of  the  Pioneer. 

Critical  Task  and  Task  Analysis:  Physical  Standards: 

The  need  to  adopt  more  rigorous  physical  standards  for  entry  into  the  Pioneer  community  was  evident  in  the 
above-mentioned  training  problems,  as  well  as  the  JAN  95  Class  B  UAV  mishap  report  that  cited  poor  visual 
acuity  of  an  EP  as  a  causal  factor  (U.S.  Navy  Message  Traffic,  1994).  Under  ideal  conditions,  many  pl^sical 
limitations  of  UAV  crew  members  may  have  no  impact  on  safe  and  effective  mission  performance.  On  the  other 
hand,  the  fleet  and  field  environments  under  which  they  have  to  operate  can  be  extreme  in  both  climate  and 
operational  pace.  For  this  study,  the  Chairperson  of  the  NOMI  Aeromedical  Advisory  Council  took  an  approach 
that  was  similar  to  that  used  for  the  devel(q>ment  of  the  Landing  Craft  Air  Cushion  Vehicle  crew  member 
standards  (Hunt,  Lirmville,  Stuster,  Schneider,  &  Braun,  1993).  First,  the  type  of  work  being  performed  and  the 
environmental  conditions  of  Pioneer  operations  were  established.  This  was  done  via  discussions  with  flight 
surgeons  and  aeromedical  persormel  involved  in  recent  UAV  mishap  investigations  and  site  visits  to  observe  the 
operations  at  VC-6  and  VMU-1.  Secondly,  the  goal  was  to  develop  a  hypothesis  of  significant  aeromedical  factors 
involved  in  these  tasks  (i.e.,  visual  acuity,  hearing  loss,  etc.).  This  was  done  in  coordination  with  some  interim 
standards  that  had  been  developed  by  the  U.S.  Marines  to  deal  with  aeromedical  issues  in  the  field.  Lastly, 
structured  interviews  with  squadron  and  unit  persormel  were  used  to  test  the  hypothesized  elements,  establish  their 
relevance,  and  make  formal  reconunendations  to  the  Aeromedical  Advisory  Council. 

The  applicable  physical  standard  elements  by  position  can  be  seen  in  Table  4.  As  would  be  expected, 
commutucation  (both  speech  and  audition)  and  visual  factors  are  important  for  all  positions,  but  most  critical  for 
EPs.  Given  that  a  large  percentage  of  Pioneer  flights  are  at  night  and  EPs  have  to  guide  the  craft  in  using  only 
wing  tip  and  tail  lights  of  the  craft,  color  and  good  night  vision  are  essential.  Because  of  the  importance  of  good 
visual  acuity,  the  recommendation  was  that  EPs  should  also  receive  trairung  on  factors  that  can  influence  acuity. 
The  final  guidelines  and  recommendations  made  by  the  Aeromedical  Advisory  CouncO  were  to  use  the  existing 
physical  examination  standards  that  are  used  for  Navy  and  Marine  Corps  air  traffic  controllers  with  an  additional 
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requirement  of  depth  perception  testing  for  EPs. 


Table  4.  Applicability  of  Physical  Standard  Elements  by  Function. 


Task 


Ejdemal  Inlerral  Payload  Mission 

Pilot  Pilot  Opetatdr  Commander 


Notes 


Night  vision 
Color  vision 


X 

X 


X 

X 


X 


External  wingtip  limits;  only  cue  during  rolling 
T/O 


Depth  perception 

X 

Distant  vision 

X 

Near  vision 

X 

X 

X 

Peripheral  vision 

X 

X 

X 

Accommodation 

X 

Hearing 

X 

X 

X 

Clear  speech 

X 

X 

X 

Stress  coping/AA 

X 

X 

Medication  use 

Motion  sickness 

X 

Physical  training 

X 

X  Rolling  T/O;  interpretation  of  turns;  console 

warning  lights 

X  Operator  75-90  ft  from  net;  mid  runway  for  rolling 

recovery 

X 

X 

X 

X  Checking  control  box  switches 

MC  role  >75%  hearing  based 
X  Reading  Aloud  Test 

X  USMC:  Multiple  taskings;  USN:  Changing 

operator/  maint  roles 

Any  affecting  vision,  hearing,  balance,  alertness, 
or  judgment 

Simulator  sickness/ship;  IP/PO  have  no  external 
references 

X  Visual  illusions  especially  at  night; 

_ fatigue/nutrition _ 


Statistical  Analysis 

Log-transformed  error  scores  (cumulative  number  of  pixels  from  the  center  of  the  screen)  were  used  as  the 
dependent  measure  on  all  CBPT  tracking  tasks  (see  Appendix  A  for  tasks  and  dependent  measures).  Number 
correct  and  reaction  times  for  correct  responses  were  used  on  the  Manikin  and  DC  tad«,  and  the  total  number  of 
correct  responses  across  trials  was  analyzed  for  the  DLT. 

Factor  analysis  was  tised  to  develop  a  single  composite  predictor  score,  and  this  score  was  regressed  on  a 
criterion  reflecting  overall  training  performance.  The  final  step,  after  choosing  the  best  regression  equation,  was  to 
locate  a  cut-off  score  to  be  used  by  the  prediction  algorithm.  The  prediction  algorithm  serves  to  classify  a  candidate 
who  desires  to  enter  UAV  training  as  either  a  predicted  PASS  or  a  predicted  FAIL  based  on  his  performance  on  the 
CBPT.  All  analyses  were  done  using  SPSS  6.1. 


RESULTS 


PREDICTORS  CHOSEN 

The  original  data  from  the  CBPT  consisted  of  63  raw  scores,  from  which  8  scores  were  derived  to  capture  the 
essence  of  the  CBPT  tasks  (see  Appendix  B).  The  derived  scores  represented  psychomotor  performance  on  (1) 
stick  only  (S);  (2)  DLT  only  (D);  (3)  stick  combined  with  DLT  (SDLT);  (4)  stick  and  rudder  (SR);  (5)  stick  and 
rudder  combined  with  DLT  (SRDLri;  (6)  stick,  rudder,  and  throttle  (SRT);  (7)  horizontal  tracking  with  digit 
cancellation  (HTDC);  and  (8)  qwtial  rotation  (MANIKIN).  A  factor  analysis  was  performed  on  the  eight  derived 
scores,  and  only  those  factors  with  eigenvalues  greater  than  1.00  were  retained.  Four  fectors  were  retained  using 
this  criterion.  When  various  regression  equations  were  evaluated  using  the  factor  scores  from  the  four  fectors  as 
indepentip^ut  variables,  the  most  parsimonious  model  included  just  the  first  factor.  Therefore,  a  single  predictor 
variable-a  linear  combination  of  the  eight  derived  p^chomotor  variables-was  retained  in  the  algorithm  to  predict 
student  success.  The  weights  assigned  to  each  derived  score  are  the  weights  that  are  used  to  construct  the  first 
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weights  are  explicitly  shown  below  in  the  section  Prediction  Algorithm 
CRITEMON  MEASURES 

The  criterion  measures  were  identified  from  the  biweekly  instructor  evaluation  sheets  and  the  actual  flight 
data.  A  criterion  score  between  0  and  100  was  assigned  to  each  of  the  trainees,  with  a  higher  criterion  score 
representing  better  performance  in  training.  Instructors  were  also  asked  to  provide  subjective  overall  ratings  of  the 
participants  on  the  same  scale.  The  ratings  were  averaged  across  instructors  for  corroboration  of  the  more 
objective  scaling  obtained  from  the  flight  data  sheets.  The  ratings  between  the  two  methods  were  highly  correlated 
(see  Fig.  4). 


Figure  4 


LINEAR  REGRESSION 

As  mentioned  previously,  a  number  of  linear  regressions  were  carried  out  on  these  data  with  either  one,  two, 
three,  or  four  factor  scores  used  as  independent  variables  and  the  criterion  score  as  the  rlRppndpnt  variable.  When 
the  from  the  linear  regression  was  penalized  for  including  extra  independent  variables,  the  regression  nring  the 
first  factor  score  was  singled  out  as  having  the  highest  adjusted  R^. 

Criterion  data  were  available  on  only  eight  external  pilot  participants,  so  three  “mock”  data  points  were 
inserted  into  the  regression  in  an  attempt  to  improve  the  predictive  algorithm.  Three  fictitious  data  points  were 
calculated  ftir  criterion  scores  at  0, 50,  and  100  to  represent  worst,  mediocre,  and  best  training  perfisrmance, 
reflectively.  This  attempt  to  anchor  the  end  points  of  the  model  (i.e.,  extremely  bad  test  performance  and 
extremely  good  test  performance)  is  similar  to  the  “mixed  estimation”  technique  (Belsley,  1991)  used  to  improve 
linear  regression  models.  See  Fig.  5  for  a  plot  of  the  first  factor  score  versus  the  criterion  score  for  these  1 1  data 
points  (i.e.,  8  experimental  participants  and  3  mock  data  points).  The  experimental  participants  are  shown  as 
circles  in  the  figure,  and  the  mock  data  points  have  the  numbers  1, 4,  and  11  above  them.  The  adjusted!^  for  the 
linear  regression  through  these  points  was  0.86.  A  linear  combination  of  the  eight  derived  psychomotor  variables 
predicted  a  high  proportion  of  the  variance  in  UAV  training  performance. 
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Figure  5 
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PREDICTION  ALGORITHM 

The  end  product  of  this  research  effort  is  a  single  algorithm  for  predicting  whether  a  candidate  is  likely  to  pass 
or  foil  UAV  training.  The  final  step  in  accomplishing  this  goal  is  the  selection  of  a  threshold  score. 

Determining  such  a  cut-off  score  depends  on  many  factors  that  must  be  made  by  the  training  community  itself. 
Therefore  we  queried  the  UAV  instructors  and  our  own  data  for  a  grade  that  represented  the  mimmum  acceptable 
performan^  for  transition  to  a  fleet  assignment.  The  UAV  instructors  chose  a  grade  of  80^  indicating  minimal 

acceptable  performance.  This  meant  that  two  of  the  experimental  participants  were  not  performing  adequately 

during  training.  The  instructors’  cut-off  score  corresponded  to  a  criterion  score  of  50,  which,  in  turn,  conesponds 
to  a  factor  score  of  0.  Simply  put,  this  allows  the  prediction  algorithm  to  be  stated  as:  if  the  factor  score  for  a 
Oj^nHiHatp.  >  0,  then  predict  PASS. 

In  detail,  this  is  calculated  as  follows.  The  predictor  variable  (or  first  factor  score)  is  computed  as: 

Predictor  variable  =  .1866zi  +  .01735z2  +  .29152z3  -  .27195z4 

+  .26090z5  -  .25166x6  +  .I86I8Z7  -  .21437z8 


This  is  a  linear  combination  of  z  scores  for  the  eight  derived  measures.  Negative  weightings  on  Z4,  Zg  (S, 

SR,  and  SRT,  respectively)  are  due  to  the  feet  that  a  higher  tracking  score  is  correlated  with  poorer  perform^ 
and  vice  versa  The  foUowing  is  the  variable  assignment  corresponding  to  the  above  equation:  zi  -  D;  zz  HTDC; 
Z3  =  MANIKIN;  Z4  =  S;  Z5  =  SDLT;  Z6  =  SR;  z,  =  SRDLT;  and  zg  =  SRT.  After  the  value  of  the  pre^ctor  viable 
has  been  calculated,  the  prediction  algorithm  can  be  invoked.  If  the  predictor  vanable  <  0,  then  predict  FAE., 
otherwise  predict  PASS. 


This  predictor  variable  is  simply  a  feaor  score,  and  factor  scores  are  defined  as  Imear  combmations  of  z 
scores  For  each  individual  candidate  in  which  a  prediction  is  desired,  a  z  score  on  each  of  the  eight  tests  is 
computed.  Each  z  score  gives  the  relative  standing  of  that  individual  for  each  of  the  eight  tests.  Tfiese  z  scores  are 
then  weighted  according  to  the  above  equation.  For  each  individual  candidate,  the  z  score  for  the  1  test  is 
computed  by  taking  the  candidate’s  score  on  the  i*  test,  subtracting  the  estimate  of  the  overall  mean  for  that  test 
and  then  dividing  by  the  estimate  of  the  standard  deviation  of  the  i  test. 
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In  symbols: 


z,= 


Xi-Xi 

SDi 


DISCUSSION 

The  DUTC  training  pro^nm  currently  trains  about  12  EPs  annually,  nidch  was  tiiie  reason  for  die  limited 
sample  size  in  this  study.  Even  so,  we  found  a  strong  association  between  the  tests  in  this  battery  and  training 
performance.  In  fact,  compared  to  other  operational  systems  developed  at  NAMRL,  this  system  has  shown  the 
strongest  initial  relationship  with  training  outcome.  The  tracking/multitask  tests  were  found  to  be  predictive  of 
both  objective  and  subjective  measures  of  success  in  the  UAV  EP  training  program,  and  these  test  were  all  included 
in  the  final  predictive  algorithm.  Their  inclirsion  was  not  that  surprising  becairse  the  tracking  plirs  dichotic 
listening  test  is  very  similar  to  the  real-world  control/divided  attention  task  required  of  Pioneer  EPs  Airing  Navy 
shipboard  and  Marine  Corps  field  operations.  The  Manikin  test  was  also  found  to  be  predictive  of  performance. 
Although  it  is  not  a  ‘pure’  mental  rotation  test  since  it  can  be  solved  through  alternate  means  (i.e.,  matching-to- 
sample,  categorization,  etc.),  it  has  been  useful  in  this  laboratory  in  predicting  success  in  diverse  curricula  such  as 
LCAC  navigator  training  and  the  Navy’s  primary  pilot  training  syllabus. 

The  TET  is  the  only  multitask  test  that  was  unrelated  to  training  performance  for  the  EPs.  It  was  specifically 
developed  to  measure  the  ability  to  judge  time/distance  information  that  is  presumably  important  for  an  EP  during 
approaches  and  departures.  Thus  for,  the  task  has  failed  to  predict  individual  diBFerences  in  training  performance. 
We  are  currently  evaluating  the  usefulness  of  the  TET  as  a  predictor  for  other  aviation  populations. 

Our  conclusions  firom  the  CBPT  data  are  preliminary  but  promising  for  the  operational  use  of  this  system.  The 
initial  algorithms  developed  from  this  work  have  been  programmed  into  the  screening  system.  Screening  of  UAV 
EP  candidates  at  NAMRL  is  plarmed  for  the  next  few  years,  and  the  data  will  be  used  for  continued  validation  of 
the  system.  In  addition  to  the  items  included  in  the  e?q>erimental  battery,  a  new  task  (i.e.,  three-dimensional 
reversal)  that  may  be  relevant  to  training  performance  will  be  added  to  the  system.  As  more  individuals  are 
screened  and  criterion  data  become  available,  the  model  will  be  refined  and  a  final  system  developed  for  transition 
to  operatiortal  irse. 

The  medical  standard  recommendations  from  this  study  have  been  approved  by  the  U.S.  Navy  Bureau  of 
Medicine  and  Surgery  (BUMED)  and  are  to  be  incorporated  into  the  new  version  of  the  Manual  of  the  Medical 
Department  (MANMED).  The  standards  to  be  used  for  the  EP  position  are  the  same  as  those  used  for  air  traffic 
controllers,  with  an  additional  requirement  for  color  vision  and  depth-perception  screening.  As  with  the  air  fraffir 
controUer  community,  medical  consultation  is  necessary  before  resuming  flight  operations  when  ill  or  on 
medications. 
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APPENDIX  A.  Psychomotor  Task  and  Dependent  Measures 


1  TASK 

SPECIFIC  ABILITY 

DEPENDENT  MEASDBE 

:  — - — - ^ - 

Performance  Test  One 

Stick  Only 

eye-hand  coordination 

accumulated  pixel  errors  over  test  duration 

Dichotic  Listening 

divided  attention 

number  correct 

Stick/Dichotic  Listening 

eye-hand  coordination  and  divided 
attention 

accumulated  pixel  errors  over  test  duration 
and  number  correct 

Stick/Rudder 

eye-hand-foot  coordination 

accumulated  pixel  errors  over  test  duration 

Stick/Rudder/Dichotic  Listening 

eye-hand-foot  coordination  and 
divided  attention 

accumulated  pixel  errors  over  test  duration 
and  number  correct 

Stick/Rudder/Throttle 

eye-hand-foot  coordination 

accumulated  pixel  errors  over  test  duration 

Performmice  Test  Two 

1  Horizontal  Tracking  (HT) 

two-dimensional  tracking 

accumulated  pixel  errors  over  test  duration 

Digit  Cancellation  (DC) 

reaction  time  and  short-term 
memory 

number  correct  and  reaction  time 

HT/Digit  Cancellation 

time  sharing,  two-dimensional 
tracking 

accumulated  pixel  errors  over  test  duration 

Peifonnaiice  Test  Three 

Manikin 

Time  Estimation  Task  (TET) 

mental  rotation  and  short-term 
memory 

two-dimensional  tracking 

number  correct  and  reaction  time 

accumulated  pixel  errors  over  test  duration 
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Appendix  B.  Eight  Derived  Psycfaomotor  Scores 


s 

D 

SDLT 

SR 

SRDLT 

,  SET 
HTDC 
MANIKIN 


log^^iSTICK)  (1) 

DLT 

~~r  (2) 

dltjstk 

log^oiSTICKDLT)  (3) 

^/log^oiSTJiDR)  X  log^oiRDRJST)  (4) 

_ DLTjSTRD 

y/log,o{SRDJDLT)  X  log^o{RDSJDLT) 

^log^oiSHr^TK)  X  log^oiSRTJtDR)  x  log^oiSRTJT)  (6) 

f  HTDCCl  ^  HTDCC2  ^  HTDCCZ  \ 
Vlogio(FT£)Cl)  \og^Q{HTDC2)  \og^Q{HTDCZ) ) 

X 

4  (8) 


{MNl.COR  -  MNIJNC)  +  {MN2.COR  -  MN2JNC) 

+  {MNZ.COR  -  MN3JNC)  +  {MN4.COR  -  MN4JNC) 
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(1)  STICK  =  STICK  error  score. 

(2)  DLT  =  Number  correct  in  Dichotic  Listening  Test. 

(3)  DLT_STK  =  Number  correct  in  Dichotic  Listening  Test  when  done  in  conjunction  with 
STICK  tracking. 

STICKDLT  =  Stick  error  score  when  done  in  conjunction  with  Dichotic  Listening  Test. 

(4)  STJRDR  =  STICK  error  score  when  done  in  conjunction  with  RUDDER  tracking. 
RDR_ST  =  RUDDER  error  score  when  done  in  conjunction  with  STICK  tracking. 

(5)  DLT-STRD  =  Number  correct  in  Dichotic  Listening  Test  when  done  in  conjunction 
with  STICK  and  RUDDER  tracking. 

SRD-DLT  =  STICK  error  score  when  done  in  conjunction  with  DLT  and  RUDDER  track¬ 
ing, 

RDS-DLT  =  RUDDER  error  score  when  done  in  conjunction  with  DLT  and  STICK  track¬ 
ing. 

(6)  SRT_STK  =  STICK  error  score  when  done  in  conjunction  with  K’JDDER  and  THROT¬ 
TLE  tracking. 

SRT_RDR  =  RUDDER  error  score  when  done  in  conjunction  with  STICK  and  THROT¬ 
TLE  tracking. 

SRT_T  =  THROTTLE  error  score  when  done  in  conjunction  with  STICK  and  RUDDER 
tracking. 

(7)  HTDCCl  =  Number  correct  in  DIGIT  CANCELLATION  task  when  done  in  conjunction 
with  HORIZONTAL  TRACKING-First  set  of  trials. 

HTDCl  =  HORIZONTAL  TRACKING  error  scores  when  done  in  conjunction  with  DIGIT 
CANCELLATION  task-First  set  of  trials. 

HTDCC2  Number  correct  in  DIGIT  CANCELLATION  task  when  done  in  conjunction 
with  HORIZONTAL  TRACKING-Second  set  of  trials. 

HTDC2  =  HORIZONTAL  TRACKING  error  scores  when  done  in  conjunction  with  DIGIT 
CANCELLATION  task-Second  set  of  trials. 

HTDCC3  =  Number  correct  in  DIGIT  CANCELLATION  task  when  done  in  conjunction 
with  HORIZONTAL  TRACKING-Third  set  of  trials. 

HTDC3  =  HORIZONTAL  TRACKING  error  scores  when  done  in  conjunction  with  DIGIT 
CANCELLATION  task-Third  set  of  trials. 

(8)  MNl-COR  =  Number  correct  in  MANIKIN  task-First  set  of  trials. 

MNIJNC  =  Number  incorrect  in  MANIKIN  task-First  set  of  trials. 

MN2_COR  =  Number  correct  in  MANIKIN  task-Second  set  of  trials. 

MN2  JNC  =  Number  incorrect  in  MANIKIN  task-Second  set  of  trials. 

MN3_COR  =  Number  correct  in  MANIKIN  task-Third  set  of  trials. 

MN3  JNC  =  Number  incorrect  in  MANIKIN  task-Third  set  of  trials. 

MN4_COR  =  Number  correct  in  MANIKIN  task-Fourth  set  of  trials. 

MN4JNC  =  Number  incorrect  in  MANIKIN  task-Fourth  set  of  trials. 
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